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This document summarizes a project that measures 
students 1 results by statistical analyses of examination data at the 
University of Linz r with special reference to the possibilities of 
institutionalizing the methods obtained. The survey included 609 
students enrolled in business studies from 1966 to 1969. Studies at 
the university are pursued in two stages r four semesters each, that 
lead up to what may be translated as half-degree and degree 
examinations. Student success was measured in terms of: (1) the 
behavior of a student intake chosen from a field of study; (2) new 
methods of analysis usable in an institutionalized system; (3) models 
designed to forecast a student's probable success or failure; {**) a 
basis for decisions to improve conditions of study, with the help of 
a simulation model; (5) an attempt to define a global r theoretical 
model for the study sequence; and (6) proposal for an 
institutionalized system for the whole of Australia to enable 
examination data to be processed by a central computer. While the 
possibility of applying the Linz model depends on the existence of 
relatively well-structured study regulations, the system might also 
be slightly amended to apply to % any university, provided that the 
student was obliged to report before the end of his studies. 
(Author/KE) 
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Note by the Secretariat 



One of the principle aims of the Second General Conference of Memtidr Institutions 
of the Programme on Institutional Management in Higher Education is to evaluate the 
methods and results proposed by the three research groups which completed their work in 
1974. A specialist was appointed for each research group to provide an independent 
evaluation of the york before the Conference. The guidelines the experts were asked to 
follow emphasized iJhe need to evaluate the scientific importance of the results, their 
practical use for the improvement of institutional management in higher education, their 
* transferability from one institution or country to another, and the comparability of the 
methods and results with those put forward by other groups working in the same field. In 
addition, the evaluators were asked, on the one hand, to summarise the results of each 
project so as to give readers who had not had the time or opportunity to read the final 
report a clear and precise idea of its content and, on the other hand, to make specific 
suggestions for subsequent analyses in the same or related fields. 

The present Report evaluates the work of Research Group No. 1: "Measuring student 
success: a systematic statistical analysis", carried out at the University for Economics 
and Social Sciences in Linz. 

The Centre for Educational Research and Innovation wishes to extend its warmest 
thanks to the author. 
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Measuring, student success ; 
A systematic statistical analysis 

Research carried out at the University for 
Economics , Linz 

Evaluator's Report 



This study was carried out by: 

MR . TRAUNMULLER , professor of computer science at the University of Linz, previously 
in c\-\rge of the data processing department of the administration, and by 

MR. K. STRIGL , psychologist, , jer on studies at the University of Linz. 

The information was processed on the relatively modest equipment available at the 
University computer centre, namely an IBM 1130 with a capacity of 16 K words of 16 bits, 
an integrated 512 K words disc unit, a card reader, a card reader/punch, a printer and 
2 interchangeable disc units each with a capacity of 512 K words. 

A: SUMMARY OF THE WORK 
•I : Definition of Aims 

On the basis of their terms of reference, the authors defined their aims as follows: 

To measure students' results by statistical analyses of examination data at the 
University of Linz, with special reference to possibilities of institutionalising the 
methods obtained. 

This aim was subdivided into 6 points: 

1 ) Detailed analysis of the behaviour of a studant intake chosen from a given field 
of study. 

2) Investigation of new methods of analysis usable in an institutionalised system. 

3) Investigation of models designed to forecast a student's probable success or 
failure . 

4) Use of study programmes as a basis for decisions to improve conditions of study, 
with the help of a simulation model. 

5) Attempt to define a global, theoretical model for the study sequence. 

6) Proposal for an institutionalised system for the whole of Austria to enable examina- 
tion data to be processed by a central computer. 
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The study mu3t only use measurable data obtained as a by-product of administrative 
procedures and will therefore exclude any studies or surveys relating to qualitative 
criteria, such as social environment, etc. 

II - BRIEF DESCRIPTION OF THE EDUCATIONAL INSTITUTION 

The University of Linz was founded in 1966/67. 

The survey relates to 609 students enrolled for business studies during the years 
1966-1967, 1967-1968 and 1968-1969, excluding those who left Linz to register at another 
university or in another field of study. 

Studies are pursued in two stages, each of 4 semesters, which lead up to what may 
be roughly translated as half-degree and degree examinations. The -first stage provides 
a general background and specialisation does not begin before the second stage. 

Half-degree examinations can only be taken during the 4th semester, and the present 
paper focuses on 4 separate examinations conducted by the professors of the subjects 
concerned. 

Degree examinations can only be taken at the end of the 8th semester, before an 
examining board covering 4 fields of study several of which offer a choice of subjects. 

In order to qualify for these examinations the student must have obtained a certain 
number of practical work certificates, and this he can do at virtually any time. Marks 
are also awarded on these certificates. 

It will be noted that students cannot attend courses in the second stage until they 
have passed the half-degree examinations. 

It may also be mentioned that the usual practice is not to send marks to the admini- 
stration unless they are satisfactory. 

Although the closing date of the survey, i.e. 31st December, 1972, fell later than 
the minimum period required for the groups of students in the survey to complete their 
studies, 145 students (i.e. 24 per cent of the population considered) had not yet fini- 
shed although they had obtained their half-degrees and 14 per cent of the students, 
while still enrolled had not yet obtained their half-degrees and were therefore con- 
sidered to have failed. 

III - DETAILED ANALYSIS OF THE BEHAVIOUR OF A STUDENT INTAKE CHOSEN FROM A FIELD 

OF STUDY: INVESTIGATION OF METHODS OF ANALYSES 

The student population in the survey , as stated above, consists of the students 
enrolled for business studies in 1966-67, 1967-68 and 1968-69, i.e. 609 students 
(Cf. 2.2). 

In aggregate , it may be said that: 

25 per cent of students obtained their degrees, 

24 per cent were still at university having obtained their half-degrees, 

51 per cent either abandoned their studies or were still enrolled but had not 

obtained the half-degree. 

From a factorial analysis covering a total of 48 variables listed in detail in 
Annex 3.7 of the original report (for example: award of the degree or half-degree, 
mark obtained for certificates and relevant semester mark obtained in examinations and 
relevant semester, number of optional certificates and average mark obtained during the 
first 4 semesters - by semester, number pf compulsory certificates obtained during the 
first 4 semesters - by semester), it emerges that the strongly correlated variables are 



the marks obtained and the semesters when the certificates were obtained or the 
examinations taken. 

In conclusion, the authors consider that success in a course of study depends both 
on the rapidity with which it is taken and on the quality of the results obtained, these 
tv/o strongly correlated factors having to be interpreted as a single joint factor 
(cf. 2.4.1). 

The analysis of the duration of studies shows that of the 255 students awarded the 
half-degree, 65 per cent obtained it in the 4th seme <er, while of the 112 students who 
gained degrees, 73 per cent graduated in the course of the 9th or 10th semester. Delays 
in the study programmes occurred mainly during the second stage of study (Cf. 2.4.2). 

The analysis of drop-outs shows that of 224 students who gave up their studies, 214 
did so during the first stage of study and that 202 students had still not even obtained 
certificates for practical work. Once the half-degree has been passed, very ^ew students 
give up (Cf. 2.4.2.4), although it is probable that some of those who did not finish 
their examinations by 31st December, 1972 will also drop out. 

The result is the following curve: 

number of 
failures 



- M I I 1 1 I 1 I I I i ) M 



semesters 

The analysis of practical work certificates shows that out of a total of 609' 
students, 196 had not obtained any certificates at all (mostly older students) and that 
most certificates were obtained at the beginning of the two stages of study (Cf. 2.4.3.1). 

After further analyses, 3 patterns of study emerge: 

1) Successful students , i.e. those who obtain their degrees in the minimum scheduled 
time; the feature of this group is its high level of activity with regard to both 
practical work - compulsory and optional - and examinations, at which good marks 
are obtained. 

2) Slower students who have to repeat more examinations and obtain less satis- 
factory results and who differ from the first groups as regards the time when 
they take their practical work certificate and sit for examinations in certain 
areas of study, and the fact that these examinations are taken later. 

3) The failures , who pass no practical work certificates or examinations and are 
sometimes eliminated in certain compulsory subjects (Cf. 2.4.4.2). 

Among the students obtaining their half-degree , the 3 groups were represented 
as follows: 

1 ) successful 1 90 

2) slower: 81 

3) failures: 229 
(Cf. 2.4.4.3) 
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The decree wa;j obtained by 62 per cent of those who were successful at the half- 
degree stage, and by 27 per cent of the slower students. 

However, 36 per cent of successful students in the first stage fell behind in the 
second stage (Cf. 2.4.4.4). 

The various statistical tests are described in detail in Chapter 3.4. Mainly be- 
cause of the small numbers involved, no attempt was made to standardize the examination 
results. 

Students who swi tch to another field of stud y may be divided into two groups: 

a) those who switch only once (wrong initial choice); 

b) those who switch often, are almost always slow students and finally drop out 
(people who simply wish to retain their student status) (Cf. 2.4.5). 

In conclusion , it emerges from the various analyses that: 

1 ) Student behaviour remains more or less constant : 

2 ) There a re indicators which enable a distinction to be made between the various 
behaviour groups, and this leads the authors to think that it should be possible 
to make an individual forecast as to the chances of success or failure fairly 
soon after studies have begun. 

Analysis of the study sequence by the cluster method (Cf. 3.5). 

Generally speaking, these methods' enable individuals belonging to a given population 
to be grouped mathematically on the assumption that an individual always belongs to the 
group closest to him. There are several methods but the advantage of the one evolved 
at Linz is that there is no need to specify a distribution function to be optimised nor 
is there any need to know the distribution of the individuals in the population before- 
hand or to fix the number of groups to be found. 

In short, the method is to demonstrate the existence of clusters of dots without 
assuming any particular distribution at the outset. 

Example : 



/ \ Cluster 1 
\ * / 



Cluster 2 
\ 



* *V / A x, Cluster 3 
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Applied t-j MO students who had obtained at least 1 practical work certificate, 
this method gives the following results : 

The population was divided into 9 clusters. An analysis of these clusters on the 
basis of their marks gives results similar to those recorded at the time the certificate 
was obtained, which means that these two criteria can be combined and the clusters divi- 
ded into high-level, medium-level and low-level tvpes. 

Level of marks 




Duration of studies 



Once these clusters have been determined, they can also be analysed according to 
other criteria, such as r.arks obtained by subject areas, choice of subject area in re- 
lation to time, and determination of the "compulsory subject areas". 

With reference to the initial assumption of 3 behaviour groups, it must be said 
that although the failures are again very clearly differentiated from the others, che 
successful and the slower students sometimes tend to merge inside the groups. 

It is also found that the students with the best marks are not the most rapid and, 
conversely, the most rapid do not obtain the best marks. 

It is also possible, from the pattern of marks, to detect groups of students of the 
"arts" type and those who have a more mathematical bent. 

Among the failures, there are groups with normal patterns of marks, except in one 
or two specific examinations. 

It is also apparent that, with one exception, the students in the first 4 groups 
obtained their certificates in a specific order, whereas in the bad groups there was no 
systematic order, whence the conclusion that "good" students organise their studies in- 
tuitively, which is not the case as far as "bad" students are concerned (cf. 2.5). 

IV. INVESTIGATION OF MODELS DESIGNED TO FORECAST A STUDENT r S PROBABLE SUCCESS 
OR FAILURE 

The assumption that student behaviour is consistent and that it is possible to make 
a distinction between behaviour groups, suggested the idea of attempting to allocate 
students to their nearest groups, given that the chances of success or failure in these 
groups are known. 

10 
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Two muiels were mi: 

a) a variance model, 

b) a friction model. 

The variance model is baaed on the cluster method in the sense that groups are de- 
fined and individuals allocated to the nearest group. The variables used to establish 
the groups cf successful students, slower students and failures are the number of com- 
pulsory certificates obtained, the marks awarded and the relevant semester, the number 
■-■f optional certificates. In one version of the model these variables were all given the 
same significance, while in another version they were weighted. In the event of a dis- 
parity in the forecasts between the two versions, a warning message is printed. 

Forecasts concerning the half-degree 

The survey covered a batch of 100 students in 1969/70 (whose real results were 
known) on the basi3 of groups determined from the data for the 1966/68 batches. The fore- 
casts were confirmed in the following proportions: 

68 per cent of forecasts carried out during the 1st semester, 

70 per cent " " " » » » 2nd 11 , 

71 per cent " " " » " 3rd 11 , 
63 per cent " " " » » » 4th " , 
63 per cent " " » » " » 5th » 

It should be noted that the behaviour group a student will belong to can be forecast 
by the 1st semester. If it is only a question of determining success or failure, the 
accuracy of forecasts increases to between 73 and 81 per cent. 

On the other hand, failures v/ere very inaccurately forecast in this example. 
According to the authors, the reason for this might be that the behaviour of the 1969/70 
students changed from that of the previous groups or that the groups were badly defined. 

Forecasts concerning the degree . These were based on the 1966/68 batches and the 
results are not so good (between 62 and 66 per cent for successful students, and 84 per 
cent for slower students). Here it would seem that the samples were too small and that 
the fact that the studies were not always completed influenced the results unfavourably 
(cf, 2.6, 1 and 3.6). 

The friction model is designed to measure the pace of studies on the assumption 
that they can be delayed by various negative factors (difficulty in adapting, etc.) 
which are expressed by a friction factor. This factor is determined on the basis of 
marks, number of certificates and relevant semester. 

As regards the half-degree, the following results were obtained: 

71 per cent of forecasts in the 1st semester v/ere confirmed 
67 per cent " 11 » " 2nd 11 » » 

72 per cent 11 " »' 11 3rd " 11 » 
72 per cent " " 11 » 4th » » « 

The forecasting of failures was more successful than in the variance model 
(41 per cent). 
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Forecasts ^jrmorniiiK thv decree are unsatisfactory (of. 2.6.2 and 3.6). 

In conclusion , the authors are of the opinion that forecasting is possible very 
soon after the beginning of studies and that the findings of t' ese two models, based on 
the feasibility of systematic and institutionalised analysis of observed data concerning 
certificates awarded and examinations passed are better than those obtained from other 
tests, particularly psychological tests. 

Forecasts can be either individual or collective. In the first case, the problem 
is to decide the form in which the information could be communicated either to the 
students themselves, or to the academic advisory service. Collective forecasting could 
be of value as a guide to changes that might have to be made to the system of studies, 
provided such information could be obtained rapidly. Another possible measure might be 
the systematic introduction of a trial semester to enable students to adapt (cf. 2.6.3). 

A comparison between the results of certain examinations and the size and rate of 
occupation of premises did not produce any meaningful results. It should be said that 
conditions at the University of Linz, which was built in 1966, are ideal and that the 
premises are invariably not used to their full capacity (a maximum of 87 per cent of 
the premises are used at any given time during the week and a maximum of 42 per cent of 
total capacity) (cf. 2.7.1). 

However, it was noted that the personality of the examiner is important . This be- 
comes obvious v/hen one compares the marking in parallel examinations where several pro- 
fessors hold orals on the same subject. Average marks and their distribution differ 
considerably according to the character and age of the examiner, the status of the course 
(compulsory or optional), etc. 

The authors do not, hov/ever, suggest any way of reducing the influence of these sub- 
jective factors (cf. 2.7.2). 

• 

V. THE USE OF STUDY PROGRAMMES AS A BASIS FOR DECISIONS TO IMPROVE CONDITIONS OF 
STUDY, WITH THE HELP OF A SIMULATION MODEL 

The following variables were used in this micro simulation model which was tested 
on students of computer science: 

number of students 

distribution of students by semester of study 

students' work behaviour 

study schedules' 

choice of study schedules 

long-term courses available 

choice of courses 

courses at present available 

present choice of courses 

features of present courses 

results 

Using the data obtained via the examination system, it is possible to determine 
statistically the parameters for the choice of the study schedules, the choice of sub- 
jects and the work beha/iour of students (examination results and relevant semester). 
Similar values can be obtained for the characteristics of subjects particularly new 
subjec ts . 
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The results covered : 

a) for each subject in the study schedule: the number of students who took the 
subject, the average semester in which the certificate was obtained, the 
average mark and the standard deviation of marks; 

b) study results according to students: successes, setbacks and failures, set- 
backs being expressed by a predetermined value. 

The results given in the original paper are intended as an illustration; the im- 
plementation of a system of this kind and the interpretation and reliability of the re- 
sults will depend on further in-depth research (cf. 3.7). 

VI - ATTEMPT T O DEFINE A GLOBAL THEORETICAL MODEL FOR THE STUDY SEQUENCE 

The factors influe ncing the study sequence may be classified as follows: 

1) Personal factors: 

a) social factors (social background, degree of social isolation, etc.) 

b) psychological factors (intelligence, personality, character, interests, 
motivations, etc.). 

2) Institutional factors: 

a) objectively measurable factors (study regulations, courses available, 
examination marks) 

b) subjectively measurable factors (teaching methods, behaviour of teachers, 
atmosphere at the university, etc.) 

3) Interactive factors: 

(contacts with the environment, contacts with teachers and fellow students, 
relations with parents, spouse, etc.). 

A number of hypotheses may be put forward regarding these factors : 

H1 : Social factors prevent a large number of potential students from going on to 
higher studies. 

H2: The decision to study or not and the choice of the field of study depend on 

social and psychological factors. 
H3: The student's success is determined by his adjustment to higher studies during 

the first few semesters. 
H4: Adjustment is determined primarily by psychological factors and secondarily 

by institutional and social factors. 
H5: If the adjustment is successful, psychological factors will determine the 

duration of studies. 
H6: The average duration of studies is determined by institutional factors. 
H7: If problems arise during studies, psychological and social factors come into 

play. If these problems are not resolved, institutional factors may cause 

studies to be broken off. 
H8: Institutional measures can be taken to prevent such problems from resulting 

in a student's failure. 
H9: The cybernetic aspect of the system entails a very long reaction time. It is 

therefore necessary for external bodies to support the introduction of any 

changes in the system. 

13 
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Note: The measurable data used here are simply those which are normally obtained as 
by-products of administrative procedures. 

This model may be pictured as a system of filters or as a model of friction in a 
pipe network, (cf. 2.8). 
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VII. PROPOSAL FOR AN INSTITUTIONALISED SYSTEM FOR THE WHOLE OF AUS'li IA TO ENABLE 
EXAMINATION DATA TO BE PROCESSED BY A CENTRAL COMPUTER 

The processing of the data for the present study was carried out with a small com- 
puter, using a data bank specially developed at Linz and written in FORTRAN IV. 
The chief source data is the university- administration. 
The data bank comprises the following sets of information: 

- students (personal data, previous studies, fixed data and changing data); 

- studies (fields of study, examinations, etc.). 

- teachers; 

- enrolments for semesters; 
~ timetable of courses; 

- premises. 

Access to this information is by way of indexes constructed on registration numbers, 
names of students and course codes. 

The system is supplemented by a file (groups and elements) and a code catalogue. 
Chaining of information is centred chiefly on the student. 
Input mainly takes the form of punch sheets . 

A data processing programme to support the analysis of administrative procedures 
has been developed at Linz. It shows the chaining of the different operations, the in- 
formation flow, the documents used and the processing times, all within the examination 
context. 

A simplified system of dialogue has also been introduced enabling the combinations 
of information retrieval keys and counts to be represented symbolically. (cf. 3.1 
and 3.2). 

The data bank for study programmes enables ^student information to be linked to 
course information. A data base of this kind should make it possible to record patterns 
of information linking the different courses by AND- OR relationships and showing the 
student's future courses, examinations and practical work. In addition there are a 
number of problems to be settled, such as alternative courses, optional course, parallel 
courses, courses appearing in different places in the same study programme 0 r in dif- 
ferent programmes, courses which are a prerequisite of admission to other courses, 
several courses satisfying the requirements of a study programme, etc. These problems 
have been partially resolved by means of chainings and various coding schemes in the data. 

This data base performs a central function in a university data processing system. 
It has, to provide the link between the study programme and the courses actually offered 
in a given period and, also to assign an examination or practical work result to a student 
with a comprehensive check as to whether all the conditions of the study programmes have 
been met. 

In addition to the operational function of this data base, it should also be used 
to plan teaching and evaluate the instruction offered against the requirements of the 
study programmes. The operational data should therefore be supplemented by key digits 
designed to describe the individual features of any given programme on comparable lines: 
e.g. the number of semesters to be spent on a scheduled course, the number of hours per 
week, practical work, the number of possible choices, etc. At the same time, similar 
key digits may be inserted in the course file, enabling all sorts of analyses and com- 
parisons to be made between different study programmes or between the courses required 
ty a programme and the courses actually available. These analyses are valuable tools 
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for the management and organisation of teaching and for optimising the student capacity 
of the university (cf. 3.3). 

Using the two data bases at Linz ds the starting point , the authors recommend the 
institution of a similar system for the whole of Austria and have drawn up a data bank 
project applicable to examinations. Details of this data bank project appear in 
Chapter 4 of the original text. 

> 

B: COMMENTS BY A NON-AUSTRIAN COMPUTER SCIENTIST 

, The few comments that follow were prompted by a number of interesting discussions 

which the author was privileged to have with his Austrian colleagues in the course of 
several meetings. They are, so to speak, the joint product of an exchange of views 
which was also intended to go beyond a purely Austrian context. 

Comment 1 ; 

It was difficult for the author to obtain a clear description of the aims assigned 
to the research group. For this reason it is not possible to review the results 
achieved by comparison with the objectives originally set, as the aims mentioned in the 
text were laid down a posteriori and have therefore necessarily been achieved. Would it 
not be desirable in future for these points to be established more clearly between the 
OECD and the research groups? 

Comment 2: 



To our knowledge there are three principles underlying the analysis of the study 
sequence . 

a) analysis of the student intakes; 

b) analysis of the study sequence within a given period of time, from a cross- 
section of the different intakes still present; 

c) retrospective analysis of the behaviour of students who have either completed 
or given up their studies within a given period of time. 

Model b) is certainly the easiest to introduce in that it requires no retro- 
spective acquisition of information and its prospective value is greatest, since it 
always uses the latest available information in respect of each intake, which makes this 
model very sensitive to any institutional changes made at a given time. In addition, 
the problem of students who switch to other studies can be easily resolved. 

Model c) supplements model b), but requires retrospective data. Admittedly, these 
data can sometimes be calculated very simply, even after the event (e.g. total duration 
of studies) and there is always the advantage of having all the students on hand, since 
they are still enrolled in the university. 

Model a) is, in our opinion, the most difficult to introduce since it requires either 
a retrospective acquisition of information for students who often are no longer enrolled 
in the university, or an extremely long wait until the last member of the intake has 
completed all his studies. As a result, the information is generally out of date at 
the time of its publication and is no longer of much use in the operational context. 
Finally, it should be pointed out that the problem of students who switch to other 
studies is extremely difficult to resolve since the intake is subdivided, like the branches 
of a tree, into a great many ramifications that cannot be quantified statistically. 
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Yet it is this last model which was chosen by Linz and it may be wondered whether 
some better justification should not have been found for this choice. It is apparent 
that practically no attempt was made to tackle the problem of students who switch to 
other studies and that at the end of the exercise 38 per cent of the students were still 
enrolled without having completed their studies, which appreciably reduces the size of 
the statistical population actually observable. 

Com t 3 : 

ie very restricted size of the statistical population observed (609 students, of 
whom 38 per cent had not yet completed their studies), when further divided into sub- 
groups, makes the statistical significance of the results observed somewhat unreliable. 
It therefore becomes desirable to apply the same models to much more extensive popula- 
tions, and thereby provide better justification for three hypotheses that seem funda- 
mental to further developments, namely: 

1) the behaviour of the students remains roughly constant; 

2) there are indicators which make it possible to identify clearly the different 
groups of behaviour; 

3) the pace of studies and the quality of results are strongly correlated and 
may be interpreted as a single aggregate factor. 

Comment 4 ; 

The "successful", "slower" and "failed" student categories are identified by refer- 
ence to a study programme, which would mean that in the absence of such a programme 
students could no longer be divided into categories. 

It therefore occurred to us to supplement the model proposed with a module that 
would make it possible to identify these three behaviour groups statistically on the 
basis of effective data. This system could then be extended to nearly all types of 
studies, provided that the student were able, if not obliged, to report more or less 
regularly from the onset of studies. 

The forecasting models would then be based on a student/student group comparison, 
rather than on a student/study programme comparison. The comparison of the behaviour 
groups thus determined with the study programmes in operation would make it possible to 
detect any distortions, and might serve as a basis for institutional decision-making in 
the future. 

Comment 5 : 

With this last assumption, it may be wondered whether the "successful'^'slower" 
distinction would be maintained in its present form or whether one would not end up, 
for example, with a ,r success"/"failure" concept, successes being expressed as a distri- 
bution of probability over time. 

Comment 6 ; 

The concepts of success and failure are defined solely in terms of whether or not 
a degree is obtained. If it were to be made institutionally possibly for a student to 
embark on a course of study having specified at the outset that he did not wish to ob- 
tain a degree (it therefore being in his interest to specify this), the compatibility 
of the model would be extended to other universities which admit external students, 
trainees, persons who simply wish to undergo vocational retraining, etc. There can be 
no doubt that a possibility of this kind would have also had an influence on the figures 
at Linz, where it is mentioned that failures in a certain batch occur among relatively 
older people who are also engaged in some professional activity. 
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Comment 7 : 

The use of the "cluster" method is original and could certainly be applied in 
studying the average behaviour of students in various institutions of higher education 
(see Comment 4). In the present report it would have been desirable to have a general 
table showing all the clusters in relation to one another. The results would certainly 
confirm Comment 5. The small number of students involved is a handicap but the method 
could be used in other institutions of higher education to establish a relationship be- 
tween failures and fields of study, for instance. 

Comment 8 : 

If the hypotheses of behavioural stability and differentiation of groups could be 
verified on a large scale, it seems intuitively evident that a student should be able 
to be assigned to a behavioural group. It also seems that the choice of a method, once 
the basic data are the same, becomes a factor of secondary importance. Might it not 
then be preferable to use the variance model rather than the friction model, since the 
former would enable comparisons to be made either with the study programmes or with the 
groups of students observed. 

Comment 9 : 

In order to prove the significance of a forecast, it would be desirable to relate 
the result of forecasts to the result that would have been obtained through a random 
effect. 

From this point of view, it seems that the results mentioned in the study are not 
as conclusive as they have been made out to be. These qualified results are certainly 
due to the smallness of the samples and we are convinced that the forecasting model 
could still be greatly improved if the basic figures were more reliable. 

Comment 10 : 

It has been clearly demonstrated that the conditions at Linz as regards premises 
are so "ideal" that the premises cannot have had any influence on the results of studies. 
On the other hand, it may be wondered whether an analysis of the marking systems of the 
different examiners might not show that some adjustments were in order; this question 
might perhaps have been taken a little further. 

Comment 1 1 : 

It should also be an easy matter to register negative examination results by ad- 
ministrative arrangements or computerisation. Knowledge of these results would perhaps 
have improved forecasting possibilities. 

Comment 12 : 

The planning and creation of the data bank compels admiration considering the limited 
material means available. 

The description of the study programmes is ingenious and this file will obviously 
fulfil an important function. 

On the other hand, it certainly seems that the students and studies system as a 
whole is relatively isolated from the other components of a university information 
system. 

The same may surely be said of the link-up with the operational systems which are 
so important for the purposes of institutionalisation. It is a pity that this admirable 
programmed system could not have remained operational beyond the period of observation set 
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for the exercise. It would have been interesting to observe the behaviour of the study 
programmes model over a number of years, assuming that the study programmes could have 
been modified from year to year, since we instinctively feel that this factor may not 
have been taken sufficiently into account. 

Comment 15 ; 

This lack of link-up with the other university information systems and with the 
operational systems is similarly apparent when it is proposed to create a model for a 
data bank covering the whole country. The author would like to warn against the pre- 
cipitate introduction of a centralised system which would inevitably necessitate the 
centralisation of a great deal of other information. If such a system were also cut 
off from the operational systems, there would be a great danger of not obtaining reliable 
information and of upsetting the whole data processing project, as was possible at Linz 
with the transfer of just one person from administration to teaching. 

A centralised system should not underrate the efforts that have to be made at the level 
of each individual university. Before putting such a system into . operation, it would 
seem more prudent to test it for two or three years in one of the universities, e.g. 
Linz or Vienna. From this standpoint, it can only be regretted that the system devised 
at Linz could not have continued to operate and been further developed. 

On another level, it might then have been seen that verification by computer of all 
the conditions for admission to examinations and acceptance of a rating is a very cum- 
bersome procedure and that arrangements should at least be made for dealing with the more 
unusual cases "manually". For it must not be forgotten that such a system has to be ad- 
justed constantly to new conditions and new decision-makers, and the maintenance of this 
official information in the computer might well become so heavy a task that it could no 
longer be carried out correctly and in the required time in an operational system as 
well . 

Comment 14 : 

The presence in this text of the detailed description of the function which the 
study programmes file can and must perform is apt to be somewhat bewildering to the 
reader, since he does not always appreciate the link between this question and the ex- 
amination system. The same is true of the simulation model. 

Conclusions \ 

It would seem at first sight that the possibility of applying the Linz model depends 
on the existence of relatively well structured study regulations. On reflection, however, 
it might also be argued that the system, slightly amended, could be applied to any uni- 
versity, provided that the student were obliged to report before the end of his studies. 

The conclusion which may be drawn is that the work should not be regarded a^s finished 
but should be pursued along the lines already mentioned: 

aplication of the system to other universities; 

- verification of hypotheses over a sufficiently large and varied statistical 
population; 

investigation, through sample surveys, of criteria not objectively quantifiable 
in an administrative system, such as social environment, conditions of study 
and so forth; 

- extension of the system to make it operational and ensure a link-up with^the other 
components of a university information system; 
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testing of the system in a university over a number of years. 

Finally, the author would like to emphasize the wealth and originality of ideas 
contained in this work and to congratulate the Linz team on the results they have 
achieved in a comparatively short space of time. 



